DOCUMENT RESUME 



ED 347 246 



UD 028 748 



AUTHOR 
TITLE 

INSTITUTION 

PUB DATE 
NOTE 

PUB TIPE 
JOURNAL CIT 



Freiberg, H. Jerome; And Others 

Turning around Five At -Risk Elementary Schools. 

Publication Series 91-8. 

Temple Univ. , Philadelphia. Center for Research in 

Human Development and Education. 

90 

25p.; Journal article reprint separately published as 
a number in a monograph series. 

Reports - Descriptive (141) — Journal Articles (080) 
School Effectiveness and School Improvement; vl nl 
p5-25 1990 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



HF01/PC01 Plus Postage. 

Achievement Gains; Achievement Tests; Comparative 
Analysis; Educational Change; Educational 
Environment; "Educational improvement; Elementary 
Education; • Elementary School Teachers; *High Risk 
Students; Interviews; * Management Systems; Minority 
Group Children; Principals; "Teacher Role; Transfer 
of Training; * Urban Schools 
Texas Educational Assessment of Minimum Skills 



ABSTRACT 

This article presents a description of an 
instructional management system called Consistency Management (CM) 
and its implementation in five urban elementary schools in Texas, 
with over 2,500 students. CM aimed to create a consistently 
implemented schoolwide consensus for teaching and learning that 
provided a self-analytical approach for teachers to examine the 
creation of a positive and orderly learning en 'ironment. The schools, 
ranking in the lowest 5 percent of Texas elementary schools based on 
the Texas Education Assessment of Minimum Skills (TEAMS) , had a 
minority composition of 94 percent, with 83 percent of students 
eligible for free or reduced cost lunch. The five CM schools 
significantly improved scores on the TEAMS from 1985-86 to 1987-88 in 
mathematics, reading, and writing. CM schools increased 17 percent in 
the percentage of students passing the TEAMS, while matched non-CM 
comparison schools decreased 2 percent. Students of CM trained 
teachers scored significantly higher than did students of non-CT 
trained teachers. Discipline referrals were reduced, and inter /lews 
with 5 principals and 19 teachers indicated that the program had 
direct transfer to the classroom. The qualitative data from the 
interviews provide a stronger understanding of the results and issues 
faced fcy principals, teachers, and students. Seven tables present 
study data, and there is a 36-item list of references. (SLD) 
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Abstract 



Thi, article reports the results of a study involving five elementary schools with a large percentage of 
minority tow socio^omic students. It provides evidence that when teachers of elementary studente who are in 
Zl— participate in determining P^.^^-^^^^^^ 
research based management system, the student outcomes improve significantly. In addition, interviews with five 
P^Pals3n3 teacbT^m the schools indicated that the program had direct transfer to classrooms. THe 
study seems to support related research in classroom management and school effectiveness. 
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ABSTRACT 



This article presents a description of an instructional management system ^ Consis- 
tency Mana^ment and its implementation in five urban elementary schools in Texas. /The 
IMoST Ure identified in 1986 by the Texas Education Agency as rankmg in the 
lo^ 5 percent of all elementary schools taking the state mandated Texas Education 

A ^ys1sW the five schools significantly improved Oieir 

TEAMS tesYscores from 1985-86 to the 1987-1988 school year » 
and writing. When compared to a matched set of elementary ^^ M C °^^ 
Management schools increased 17 percent in the percent passing Ae T^Swhilethe 
non-program school, decreased 2 percent in the percent passing during ^V987-1988 
school^ar. When the students of teachers trained in the pmgramweie con^dwnh 
^ untmined teachers . based on the Metropolitan Achievement T«« (^TO and 
the TEAMS, the program (experimental) group scored significantly higher beyond the 
^OllevelinTotal language, total reading, social studies, science and total mathematics 
^ and^iting on the TEAMS. Additionally, discipline referrals were 

reduced and a series of structured interviews of the five principals and nmeteen teachers 
indicated that the program had direct transfer to the classroom (Freiberg, Prokosch, Treis- 

"'^^tr^^suppon the pioneering works of 

Schweitzer, and Weisenbaker (1979); Brookover and Lezotte (1977); Edmoncb ( 1979a 
1979b. 1979c); Goodlad (1983a. 1983b); Rutter, Maughan. Mortimore Ouston, and Smith 
1979 - Murnane (1975); Summers and Wolfe (1975); Stalling* and Mohlman (1981), 



This article is based on a paper presented at the annual meeting of the American Educational 
Ssea^Asso^Son in S^Frandsco. California in March of 1989. This paper would not 
£S without the absolute dedication and belief by the teachers and principals of 
the five schools, and by the central administration that all students can excel. 

Correspondence: Prof. H. Jerome Freiberg. College of Education, Department of Curriculum 
and Instruction, Houston, Texas 77204-5872, USA 

Manuscript submitted: June 1. 1989 

Accepted for publication: November 2, 1 989 .« 
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Lcvinc and Staric (1982); Edmonds and Frederiksen (1978); and Wynne (1980) who estab- 
lish that schools can make a difference in academic achievement regardless of socio- 
economic status. This is given substantial support from several syntheses and reviews of 
school-level impact on students* academic achievement (Benbow, 1980; Bridge, Judd, & 
Moock, 1979; Centra & Potter, 1980; Edmonds, 1979c; Glassman & Biniaminov. 1981; 
Good & Brophy, 1986; MacKenae, 1983). 

TTiis study disaggregated the school achievement data to determine the influence of 
training on teachers who were inservked in the program against those teachers from 
matched schools who were nc< trained in this specific program but received other similar 
services from the district Qualitative data derived from structured interviews of principals 
and teachers from the five schools provided a broader understanding of the results and 
issues faced by principals, teachers and their students. 



SCHOOL DEMOGRAPHICS 

The five elementary schools which were program schools are being called Washing- 
ton, Adams, Jefferson, Madison and Monroe for the purpose of anonymity. The 
schools had three primary commonalities: 1 ) the students were from minority groups 
(94 percent), 2) the students were from low socio-economic backgrounds (83 percent 
were on free or reduced lunch), and 3) student mobility was twice the average of the 
highest perforating elementary schools in the district. The student populations of 
three schools were exclusively Black, one was Hispanic and Black and one was 
Hispanic, Black and Asian. Geographically the schools were diverse with only 
Jefferson and Madison located in the same district area of the city. The percentage of 
new teachers varied from school to school. For example, Adams Elementary school 
had 60 percent of the teachers in their first, second or third year of teaching while 
Monroe elementary school had 58 percent of the teachers with eleven or more years 
teaching experience. The district average is 47 percent with eleven or more years 
teaching experience (see Table 1 for demographic comparisons between program 
and non-program schools). 



CONSISTENCY MANAGEMENT 

The Consistency Management program (Freiberg, 1983), was used as the interven- 
tion in each of the five schools. Consistency Management was the name given to the 
program which translated research in classroom management, instructional effec- 
tiveness, school climate, school effectiveness and staff development into practical 
classroom and school applications (see Ernmer & Evertson, 1981; Kounin, 1970; 
Doyle, 1986; Cohen, 1983; Freiberg, in press; Mumane, 1975; Stallings & 
Mohlman, 1981; Teddlie, Stringfield, & Desselle, 1985; Good & Brophy, 1986; 
Lightfbot, 1983; Freiberg, Driscoll & Knight, 1987; Freiberg, Buckley & Townsend, 
1983). 
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Tabic 1: Demographic comparison of program and non-program schools. Based on 1985-86 
Data. _ 

Total Ethnicity* Free/ Mobility Atten- Achievement** A<min- 
Koth redded dance Above Below istratwe 

ment Lun^ 

Percent Percent Percent Percent 



Area 



Washington 
Able*** 


517 
826 


96 
75 


3 

23 


1 

1 


98 
98 


32 
62 


OK 

94 


Adams 
Baker*** 


276 
265 


36 
37 


57 
63 


7 
0 


88 

97 


75 
44 


96 
98 


Jefferson 
Charles*** 


499 
630 


100 
100 


0 
0 


0 
0 


74 
84 


30 
41 


97 
95 


Madison 
Dillon*** 


516 
1288 


99 
99 


0 
1 


1 

0 


70 
97 


55 
53 


95 
95 


Monroe 
Edward*** 


774 
408 


43 
25 


30 27 
39 36 


85 
57 


49 
45 


96 
96 



57 
61 

28 
74 



51 
51 



55 
48 

59 
60 



21 
2 

15 
11 

10 
14 

8 
14 



• Bl = Black; His = Hispanic: Oth = Other (American, Indian. Asian, & While). 
Above or Below grade level on the ITBS Composite. 

*** non-program schools. 



12 
12 

4 
4 

4 
4 

14 
14 



The goal of Consistency Management is to establish a supportive, safe, and orderly 
classroom and school environment which is contingent upon teacher consistency^ 
Consistency begins with a cooperatively developed educational ™ th « 
grounded in shared values by members of the learning community. The goal i to 
create a school-wide consensus for teaching and learning which is consistently im- 
plemented. Consistency Management is designed to prevent problems which might 
Ere with learning, eliminating or reducing the need for more costly and time 

consuming interventions. in 
Consistency Management provides a self-analyt.cal approach for teachers to 
examine how they establish a positive, orderly learmng env.ronment. Thts was 
acS through a combination of strategically timed staff development workshop . 
"eacher and principal self-assessments, team building (students, teachers, pnnc.pals. 
and parents) and parental and community involvement. 

Instruments were provided throughout the workshop sess.ons for the school fac 
ultv tHonitor their own consistency, individually, as a faculty, and w.th the.r 
SncfpaTr^nsistency Management Calendar was provided for teachers to >den- 
rtZZ in which they were inconsistent (see Figure 1). A thirty item Consistency 
Checklist was used to review key areas identified in the workshops. Teachers audto- 
upS 3W0 Segments of their classrooms and used a Urn Inference Self-Assessmen, 



8 



HJ. FREIBERG ET AL. 



Measure (Freiberg, 1987) to code, analyze and critique their teaching and meet with 
other teachers to share ideas for instructional improvement 



WEEK OF 



FOCUS MON. TUE. WED. THUR. FRI. 



RECOGNIZE 

STUDENTS 

BYNAME 












CALL ON STUDENTS 
RANDOMLY USING 
3X5 CARDS 












PROVIDE A FOCUS 
FOR EACH LESSON 












POST STUDENT WORK 
ON THE BOARD 












POST M A W WORK TO 
GIVE STUDENT A 
MODEL OF QUALITY 












SEND HOME A 
WEEKLY CALENDAR 
WITH ASSIGNMENTS 
AND OTHER CLASS- 
ROOM INFORMATION 












BUILD STUDENT 
RESPONSIBILITY BY 
USING AN ABSENCE 
PACKET FOR 
MISSED WORK 













Source: H. J. Freiberg. (1989). Consistency Management Professional Development Pro- 
gram Materials © 1989. Used by Permission. 



Figure 1. Consistency Management Calendar 

Each of the five schools reduced the time spent on disciplining students. Adams 
Elementary School, with 276 students, had the lowest enrollment of the five schools 
and had the most problems in discipline. In 1986, 109 of the 276 students were sent to 
the office for disciplinary action, 34 students had warning notices sent home and 24 
students were suspended. In the 1987-1988 school year 19 students were sent to the 
office for disciplinary reasons. Nine of the nineteen referrals were from substitute 
teachers (Freiberg, Prokosch, Treister, Stein & Opumi, 1989W. One warning letter 
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was sent home for an after school (on the way home) off-campus fight that was 
reported to theprmcipal and nostudents were suspended. It is important to note that 
the second half of the school year had the highest proportion of misbehaviors in both 
years. 



INSERVICE SCHEDULE 

The training began in April of 1987 with a voluntary all day Saturday workshop. 
Approximately 80 percent of the teachers and all the principals from the five elemen- 
tary schools attended. A one-hour follow-up session was provided in May at each 
school site. The teachers proposed another workshop before school began. Two 
workshops were provided in August of 1987. The first August session was directed 
to aU new teachers hired over the summer and any teachers who were unable to 
attend the April session. Tht second August workshop was tor all the teachers and 
principals and the goal was to plan for the opening of school and establish a consis- 
tency plan for the year. Over ninety percent of the teachers attended either the April 
or August workshops. Participation was voluntary for all sessions. The teachers re- 
ceived State approved Advance Academic Training Credit (AAT) which could be 
used by the teachers for advancement on the Texas Career Ladder. 

Six two-hour follow-up workshops were provided at the five schools for a total of 
30 workshops. Workshops began in September and continued each month until Feb- 
ruary of 1988. The staff development design was based on previous research on the 
factors which caused staff development to transfer to the classroom in the same 
district (see Freiberg, Townsend & Buckley, 1982). 

Management Values 

The teachers were asked to examine their own management values both individually 
and coUectively. They were asked to determine how they communicate their values 
to the students and to determine where teacher-student values are either congruent or 
in conflict The Consistency Management program emphasized resolution of prob- 
lem areas. For example, inappropriate student language was a common value conflict 
area. Some students would use four letter words as if they were conjunctions for each 

, sentence. Teachers were asked to explain that language which may be used with 

friends or at home may not be appropriate in school or at work. Students were given 

1 a new context for language usage. Rather than trying to negate a home value upon 

which the teacher or school had little control student language was placed m a 

I differentiated context. 

Workshop Contents 

Workshop #7 . The first workshop, held during the first week of school, placed strong 
emphasis on contacting parents at the beginning of the year when problems were at a 
minimum. Positive Post Cards were sent heme supportir. umely return of home- 
work or being punctual to school. Teachers called home (when parents had tele- 

o {% 
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phones) to provide a portrait of the school day and strategies for parental assistance at 
home for their children. A telephone script was provided during the workshop for 
teachers who were uncertain about parental contacts. Strategies were provided for 
teachers to help students feel part of a learning community. Teachers went around the 
school taking pictures of students involved in academic tasks. Pictures of students 
reading, working in groups, studying, presenting information were posted on a bulle- 
tin board in the main hallway of the school. 

Workshop 02 emphasized the need for developing a range of questions to challenge 
student thinking. The teachers were provided a range of questioning strategies with 
an emphasis cm higher level questioning techniques. The teachers were shown video 
examples of other teachers using individual student chalkboards to help in assessing 
student learning. The video teacher would ask students, for example, a math problem 
and the class would write their answers on their own chalkboards. On cue from the 
teacher all the students would hold up their answers. Only the teacher would see their 
responses. The teacher would make a mental note of which students missed the prob- 
lem and work with them during seat work time. It enough students missed the prob- 
lem the teacher stopped and retaught the concept 

Workshop 03. Hie third workshop asked teachers to be reflective about their instruc- 
tion. Each teacher audio-taped a 30 minute segment of their classroom and used a 
Low Inference Self-Assessment Measure (see Freiberg, 1987 for a detailed descrip- 
tion of the instrument) to analyze their interaction with their students. The teachers 
listened to their tape and then prepared a written critique which they discussed 
among their peers. The principals who attended most of the sessions did not stay for 
the teacher self-assessment sessions. 

Workshop 04. The research on cooperative grouping (Slavin, 1 983) and peer tutoring 
(Berliner & Casanova, 1988) support these strategies as powerful learning tools, 
particularly for low achieving students. The fourth workshop provided examples of 
how to develop and manage these learning strategies. Many of the teachers had prior 
training in both strategies but had difficulty in managing the student focused activi- 
ties. The sessions included ways of gradually developing cooperative learning 
groups, starting with groups of two and building slowly to groups of four over a three 
month period. 

Workshop 05. The fifth workshop was conducted by an expert teacher from the 
district who was trained in Consistency Management and had extensive experience 
in learning centers. She brought her own classroom centers to the workshop and 
allowed the teachers to participate in her centers and assisted them in building their 
own math or reading center. Centers and their management were presented as a 
means of moving away from teacher and student-dependency on worksheets. 

Using centers as part of reading time allowed students to circulate from reading 
with the teacher to seat work to center work. This flow of instructional activities 
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reduced the amount of time a child would spend on any one task to about 20 minutes , 
reducing boredom and ultimately off-task stucent behavior. 

Workshop #6. The final workshop was held in February of 1988. Hie session in- 
cluded teachers reporting on the on-task/bff-task seating charts and interactive in- 
struction observations conducted by teachers' peers during the previous week (see 
Stalltngs, Needles, & Sparks, 1987 tor examples of the instruments). The teachers 
had a great discussion about the types of interactions they had with their students and 
how best to create a relaxed but productive classroom. 

As mentioned, the Consistency Management training was provided at the end of 
the previous year. This enabled the teachers to test some of the ideas with their 
current students. Although student and teacher interaction patterns were set by the 
end of the school year, teachers indicated seeing enough of a change in their students 
in the spring of 1987 to begin planning during the summer for the next school year m 
September. The before-school workshop in August reviewed and reinforced the pre- 
vious sessions and focused their attention on the importance of consistency in the 
classroom and throughout the school. 

Inservice Framework 

Too often, inservice sessions provided a few days prior to the opening of school 
present sessions on new curriculum, reading texts, different instructional approaches 
or organizational procedures that require significant changes in teacher planning and 
implementation. Few teachers are effectively able to assimilate and implement these 
new requirements as their attention is being drawn to the opening of school and the 
influx of new students. The inservice strategy for this program was designed to pro- 
vide ample time for assimilation (from April to August 1987), opportunities for col- 
legia! and administrative interaction (April, May and August sessions), and follow 
through with additional sessions for the first six months of school (September 1987 
to February 1988). 



ACHIEVEMENT ANALYSIS 

Method „ 
There were three levels of analyses conducted to investigate the effects of Consis- 
tency Management (CM) on student achievement. Level I involved determining 
whether individual school level achievement improved from 1986 to 1988 (pre-post 
intervention). Level II investigated whether aggregated student achievement im- 
provement across the five program (CM) schools was different from aggregated 
student achievement improvement across five comparison schools who received no 
training in the Consistency Management program. Level III identified the teachers in 
the program schools who had been trained in Consistency Management and com- 
pared the achievement of these teachers' students to the achievement of a random 
sample of students in the comparison schools. . ^ 
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Instruments 

Student test data were obtained on two achievement instruments: the Metropolitan 
Achievement Tests (MAT6), a nationally standardized, norm-referenced academic 
achievement battery; and the Texas Educational Assessment of Minimal Skills 
(TEAMS), a stale developed and mandated, criterion-referenced basic skills battery. 

The MAT6 are overall measures of achievement in the five basic content areas of 
language arts, reading, social studies, science, and mathematics. As nationally 
named tests, their content should be representative of the curricula being taught in 
these content areas in schools across the nation and test performance of students ina 
school or program can be compared to the performance of typical students of the 
same age and grade across the nation. The reported KR-20 reliability coefficients for 
the various batteries of the MAT6 are generally high, ranging from .63 to .98. 

The TEAMS tests measure the minimum competencies in mathematics, reading, 
and writing that are expected to be mastered by all students in specific grades. These 
m i nimum competencies were defined by die Texas State Board of Education and 
were enacted into State law in 1984 under House Bill 72. The domain of knowledge 
and skills on which TEAMS test items were developed is much narrower and more 
focused on lower-level thinking processes than the domain on which MAT6 test 
items were developed. Reliability data have not been reported for the TEAMS. 

The MAT6 are administered annually at all grade levels during the spring semes- 
ter (first administered in 1 987) to students in grades one through nine. Test scores for 
the study were obtained on the five content area batteries (Total Language, Total 
Reading, Social Studies, Science, and Total Mathematics) for the 1987 and the 1988 
admimstrations for students who were in grades two through five in 1988 in the 
program (trained in Consistency Management) and non-program groups. 

The TEAMS is also administered annually during the spring semester (first ad- 
ministered in 1986) but it is administered only in first, third and fifth grades. Test 
scores on the three content area subtests (Writing, Reading, and Mathematics) from 
1986 and 1988, were obtained for program and non-program students who were in 
grades three and five in 1988. 

Level I Analysis 

The official district evaluation of the staff development programs, including the 
Consistency Management program, which were provided to the five "low perform- 
ing** schools to help the teachers in these schools improve student achievement, 
included an analysis of the simple raw change in scale scores on the TEAMS for 
students with 1986 and 1988 test data. Correlated sample t tests were used to deter- 
mine whether there was statistically significant improvement in the mean scale score 
from 1986 to 1988 at each of the tested grades, on each of the subtests, at each 
program school. 

While all five schools showed statistically significant improvement in test per- 
formance on the TEAMS in one or all subtests in one or both grades, this level of 
analysis should be viewed as a very cursory preliminary investigation. The multiplic- 
0 ity of the t test comparisons without adjusting the overall alpha level simply in- 

ERIC , i 



TURNING AROUND FIVE AT-RISK ELEMENTARY SCHOOLS 



creases the chances of making a Type I error. More importantly, no comparison was 
attempted between the program schools* student performance and a comparable set 
of non-program schools' student performance. 

Level II Analysis , 

In this second level, five schools were selected to serve as a comparison group of 
schools to the program schools. The comparison schools were matched as closely as 
possible, to the five program schools on the following variables (based on 1985-86 
data in which the five program schools were identified as "low achieving schools): 
total' student enrollment, district administrative area, student ethnic composition, 
student mobility and attendance rates, student free/reduced lunch rates, and student 
achievement (i.e., percent of students above and below grade level as determined by 
performance on the Iowa Tests of Basic Skills, administered during the spring of 
1986). Table 1 provides the data regarding these variables for the five paired sets of 
schools and Tables 2 and 3 present the scale scores. 



Table 2: Teams mean scale score. Matched Students 



Math 
1-3 3-5 



Reading 
1-3 3-5 



Writing 
1-3 



3-5 



Program schools 



Washington 

Adams 

Jefferson 

Madison 

Monroe 



88 
86 

88 
86 

88 
86 

88 
86 

88 
86 



817.96 N 
787.45 (59) 

726.36 
769.72(11) 

855.17 
801.02 (45) 

808.29 

833.97 (44) 

802.86 
779.41 (29) 



736.36 
726.89(46) 

730.90 
743.90 (20) 

802.17 
699.69 (39) 

737.15 
680.53 (45) 

799.87 
732>i2 (47) 



750.20 N 


746 91 N 


742.62 


775.84 N 


706.62(59) 


770.47 (46) 


734.00 (59) 


656.63 (46) 


675.45 


766.55 


721.81 


753.90 


668.00 (ID 


731.55 (20) 


SSU)O(ll) 


680.70 (20) 


101.31 


743.61 


776.33 


739.46 


'73.06(15) 


695.35 (39) 


754.44 (45) 


700.02 (39) 


755.15 


724.77 


747.63 


737.55 


760.11 (44) 


674.88 (45) 


786.04 (44) 


625.93 (45) 


:'50.17 


783.53 


698.10 


767.34 


1*5.51 (29) 


730.74 (47) 


76175 (29) 


654.14 (47) 



Non-program schools 



O 
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88 707.01 

Able 86 779.27(59) 

88 791.82 

Baker 86 849.21 (23) 

88 762.25 

Charles 86 791.86(58) 

88 728.57 

Dillon 86 808.23(122) 

88 796.82 

Edward 86 794.42(35) 



754.80 
695.21 (51) 

787.05 
796.40 (20) 

730.61 
679 36 (52) 

696.64 
661.71 (111) 

813.26 
778.43 (30) 



680.37 
769.74 (59) 

769.34 
722.65 (23) 

708.62 
752.96 (58) 

704.88 
731.72(122) 

782.85 
768.25 (35) 



741.70 
712.84 (51) 

771.65 
757.35 (20) 

728.50 
683.57 (52) 

728.84 
690.18(111) 

815.36 
742.03 (30) 



645.52 
774.25 (59) 

751.08 
734.95 (23) 

710.32 
788.79 (52) 

686.4 

726.4 (122) 

735.60 
735.60 (35) 



709.11 
691.27 (51) 

711.40 
690.80 (20) 

683.90 
653.88 (52) 

670.79 
668.31 (HI) 

814.56 
713.83 (30) 
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Table 3: Difference in teams mean scale score. 1988-1986 Matched Students 





3-1 


Math 


Reading 








5-3 




JO 












Program schools 








Washington 


30.51 


9.47 


43.58 


-23.56 


8.62 


119.21 


Adams 


-44.36 


-13.00 


7.45 


35.00 


40.81 


73.20 


Jefferson 


54.15 


10148 


28.25 


48.26 


21.89 


39.44 


Madison 


-25.68 


56.52 


-4.96 


49.89 


-38.41 


111.62 


Monroe 


22.65 


67.45 


64.66 


52.79 


-28.65 


113.20 



Non-program schools 



Able 


-72.26 


59.59 


-89.37 


28.86 


-128.73 


17.84 


Baker 


-5739 


-9.35 


46.69 


14.30 


16.13 


20.60 


Charles 


-29.61 


51.25 


-44.34 


44.93 


-78.47 


30.02 


Dillon 


-79.66 


34.93 


-26.84 


38.66 


-40.00 


2.48 


Edward 


2.40 


34.83 


14.60 


73.33 


-90.00 


100.73 



The investigation in this level still focused on basically school level data. In this case 
a comparison was made between the set of program schools and the set of non- 
program schools on the difference from 1986 to 1988 in the total percentage of 
students who passed one or more of the TEAMS subtests. The overall percentage of 
students who passed the TEAMS increased from 61% to 78% in the program 
schools, while the percentage of students who passed in the non-program schools 
decreased from 63% to 61 %. (SeeTable 4 for the individual school percentages). The 
McNemar test was used to test whether the passing proportions were identical from 
pre- to posttest within each group. The program group's increase of 17% more 
students passing was statistically significant (2 = 14.1, p < .001), while the decrease 
of 2% fewer students passing in the non-program schools was also significant (2 = 
5.7, p<. 001). 



Table 4: Teams aggregate % passing. Matched Students Pij* r»j* 





Program schools 




Non-program schools 




1985-86 


1987-88 




1985-86 


1987-88 


Washington 


58.3% 


80.2% 


Able 


64.8% 


55.2% 


Adams 


58.1% 


73.1% 


Baker 


73.8% 


80.5% 


Jefferson 


68.3% 


82.6% 


Charles 


61.8% 


61.4% 


Madison 


56.6% 


70.8% 


Dillon 


57.6% 


54.5% 


Monroe 


60.9% 


78.0% 


Edward 


71.3% 


79.9% 


Total 


60.06% 


78.0% 


Total 


62.7% 


60.9% 



o 'where Pij = # of passets for subttst(i) gradeO). nii = # of took subtest© grade®. 
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These results indicated that something was working in the program schools - student 
achievement on the TEAMS was improving. Thus far, though, we had been looking 
at only raw change, only at one measure of achievement (a measure that was not 
necessarily representative of overall student achievement), and only at a global 
school performance base. 

The inclusion of a third level of analysis reflects a growing concern about the lack 
of stability of achievement as reported by standardized test scores for improving 
schools. The aggregation of data may mask groups of students who continue to 
perform poorly but whose scores are lost in school-wide reporting of achievement 
(Freiberg, in press). 

The research directors of member school districts in the Council of Great City 
Schools at their annual meetings have expressed continuing concerns about the 
inconsistencies observed in schools that have made gains in test scores only to 
regress after a relatively short period of time (Frechtling, 1987). The Level I & II 
types of analyses described in this paper are acceptable procedures followed by many 
large school districts with permanent research and evaluation departments. Given 
the multitude of programs in large urban school districts and the general lack of 
resources for evaluation, let alone research, completion of Level I and II analyses 
would be considered more than necessary. Based on the lack of stability of test scores 
reported nationally tor improving schools in other districts school improvement 
efforts, an additional level of analysis was conducted to examine program effective- 
ness. 

A third level of analysis was conducted to control for pretest effects on posttest 
performance, to include a broader base academic achievement measure, and to com- 
pare the performance of students who had been instructed during the 1987-88 school 
year by teachers actually trained in Consistency Management to the performance of 
students who had been instructed by teachers not trained in Consistency Manage- 
ment in the non-program schools. 

Level III Analysis 

Subjects 

Teachers from the five program schools who had attended at least seven hours of 
Consistency Management training during the spring and/or summer of 1987 were 
identified (n = 1 37). Teachers who met the following criteria were then selected from 
this group. The teachers had to: 

1 . be a regular classroom teacher (grade 2, 3, 4 or 5), 

2. have a readily available computer file identification number, and 

3. have completed the 1987-88 school year in the same program school that he/she 
began the school year. 

Twenty-eight teachers met these criteria. Students of these teachers were then identi- 
fied on the master computer data file. 

Students were randomly selected from the five non-program schools according to 
the grade distribution of the students identified in the program schools. The total 
number of students (n = 335) sampled from the comparison schools did not exactly 
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equal the total number of students (n « 364) included in the study from the program 
schools because of insufficient numbers of students with complete test data. Students 
in the two groups were fairly similar as to ethnic breakdown and percent on free/ 
reduced lunch (Table 5). These were the only two demographic variables that could 
be checked readily from the computer file at the individual student level. It should be 
noted that the comparison schools in which the student., were randomly selected 
were matched for five additional comparable variables. 



Table 5: Student sample. 





Black 


Ethnicity 
Hispanic 


Other 


Free/reduced lunch 






percent 




percent 


Program students 


90 


80 


2 


72 


Non-program students 


85 


9 


6 


79 



Procedure 

A one-factor (two-level) multivariate analysis of covariance (MANCOVA) was per- 
formed on the five batteries of the MAT6 to determine whether achievement differed 
between program students (students who were instructed by teachers with Consis- 
tency Management training) and non-program students (students who were in- 
structed by teachers without Consistency Management training). The mean NCE 
(Normal Curve Equivalent) scores on each of the five MAT6 batteries from the 
spring 1988 administration, which represented posttest performance, were the set of 
dependent measures which were adjusted by the set of mean NCE scores from the 
1987 administration, which represented the pretest performance. Hie adjusted 
posttest mean differences between the groups were analyzed. 

The MANCOVA model makes it possible to examine academic achievement ho- 
listically, taking into account the correlation in performance across the different 
content areas. This model also allows for the statistical adjustment of posttest per- 
formance by pretest performance. 

The following assumptions of the analytical model were tested and met: 

1 . Linear relationship among the dependent variables 
(Bartlett Test of Sphericity ■ 846.94124, #=10, p <.001) 

2. Linear relationship between the set of dependent variables and the set of covari- 
ates 

(Wilks lambda = .19347, F = 56.88452, 25, 2557 31, p <.001 ) 

3. Homogeneity of the regression hyperplanes for the ccv ariates for each group; 
(i.e., there was no pretest by group interaction effect) 

(Wilks lambda = .95898, F = 1.15152, df= 25,2538.73, p =.274) 1 5 



TURNING AROUND FIVE AT-RISK ELEMENTARY SCHOOLS 



The assumption of homogeneous variance-covariance matrices between groups was 
not supported. Cheater variability was associated with the non-program group of 
studems?As this group was the smaller of the two groups, the statistical test then 
becomes a more liberal test (Stevens.1986). This liberality can ^ counterbalanced 
by testing statistical difference at a stricter alpha level (Stevens, 1 986). Alpha for this 
study was set at .01 instead of the more traditional .05 level. 

A second one-factor (two-level) multivariate analysis of covariance was per- 
formed on the three subtests of the TEAMS tesL Mean z-scores (student scale scores 
on each of the subtests were converted to z-scores within grade level) from the spring 
1988 administration (posttest variables) were adjusted by the set of mean z-scores 
from the spring 1986 administration (pretest covariates). The adjusted posttest means 
were then analyzed. 

All assumptions that were tested were substantiated. 
1 Linear relationship among the dependent variables 
(Bartlett Test of Sphericity = 1 67.20875, # =3, p <.001 ) 

2. linear relationship between the set of dependent variables and the set of covan- 

ales __ <x 
(Wilks lambda = .58627, F « 17.771749, 9, 649.96, p cOOl) 

3. Homogeneity of the regression hyperplanes for the covariates for each group; 
(i.e., there was no pretest by group interaction effect) 

(Wilks lambda = ..97557, F = .72936, 4f= 9, 642.66, p =.682) 
4 Homogeneity of the dispersion matrices 

(Boxs M = 14.49899, F - .67424, 21, 271990, p = .863) 



,ERJC 



The results of the multivariate analyses on the MAT6 and TEAMS achievement 
instruments indicated that statistically significant differences in adjusted posttest 
means existed between program students and non-program students In each analy- 
sis, the set of adjusted posttest means for students of program teachers was higher 
than the set of means for students of non-program teachers. 

(MAT-6: Wilks lambda = .92776, F = 10.71428, #=5, 688, p <.001; TEAMS. 
Wilks lambda = .82880, F = 18.38451, #= 3. 267,p <.001) 

The univariate F tests on the individual batteries of the MAT6, indicated that 
achievement was greater for the program group of students on Total J-«« V < 
.001). Total Reading ip = .009), Social Studies ip < .001), Science ip = .007) and 
Total Mathematics (jx .001).(Table 6). , L 

The subsequent univariate F tests on the individual subtests of the TEAMS indi- 
cated that achievement was greater for the program group of students on the Writing 
subtest (p < .001 ) and on the Mathematics subtest ip < .001 ). No statistical difference 
existed between the two groups on the Reading subtest ip= .21 1 ). (Table 7). 

The TEAMS tests were used as the basis by the State for identifying the five 
schools in 1986 as the lowest performing elementary schools in the dismct and are 
used by the local media for in-district as well as between-district comparisons for 
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Table 6: Pre-, post-, and adjusted posttest mean NCE's on MATS, Grades 2. 3, 4, & 5. 



Observed pretest Observed posttest Adjusted posttest 
Means Means Means 



Total language 
program 
non-program 


48.3 
46.7 


53.1 
46.3 


52.4 
47.1 


Total leading 
program 
non-program 


42.2 
40.7 


43.6 
40.0 


42.8 
40.8 


Social studies 
program 
non-program 


40.6 
39.1 


47.2 
41.6 


46.6 
42.3 


Science 
program 
non-program 


40.7 
38.3 


47.3 
42.9 


46.5 
43.7 


Total mathematics 
program 
non- program 


51.2 
48.2 


55.1 
46.7 


54.0 
47.8 


Table 7: Pre-, post-, and adjusted posttest mean z-scores on teams. Grades 3 & 5. 




Observed pretest 
Means 


Observed posttest 
Means 


Adjusted posttest 
Means 


Writing 
program 
non- program 


-.06140 (735.1) 
.06230 (741.6) 


.30238 (781.2) 
-.30684 (726.3) 


.32088 (783.3) 
-.32560 (724 2) 


Reading 
program 
non- program 


-.06093 (712.5) 
.06183 (728.0) 


.04032 (734.5) 
-.04091 (726.8) 


.07182 (737.2) 
-.07187 (724.1) 


Mathematics 
program 
non-program 


.03252 (705.9) 
.03300 (717.2) 


.23218 (761.9) 
-.23559 (689.9) 


.25464 (763.8) 
-.25838 (688.0) 



Mean scale scores are provided in 0 for information only. 
Program n = 176 
Non-program n « 166 




student achievement. Given the lack of available reliability and validity data on the 
TEAMS, the MAT-6 data in comparison represents truer measures of academic 
performance; and, therefore, one may have greater confidence in the analysis of these 
data. 1 7 
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Limitations 



There were a variety of programs directed toward students and teachers in the five 
schools. Sorting "trough each program and trying to determine .its parucul* _effcc 
on student achievement is a difficult task given the mtenelauonstap amo^acuvmes 
within a school. One of the principals clarified this issue when she "atedjto Con- 
sistency Management program was the glue which held the other programs to- 
gether." The other principals concurred with this assessment 

The results of these analyses provide preliminary support to the premise that 
teachers who are trained in Consistency Management and utilize the principles in the 
classroom can facilitate improvement in student academic achievement 

There are, of course, limitations to this study, as there are with any study m a 
natural school environment It is always difficult to assess program effect in schools 
because of the multiplicity of contributing factors to student achiejmenttha 
abound. The final level of the study, however, did compare the test performance o 
only those students whose teachers had been trained in the program to the test 
performance of students whose teachers had not been trained and who were : n 
SaXscbool environments. Student predisposition to testing ww connoUedto 
aTextent by removing from the posttest the effects of the pretest and by comparing 

these adjusted posttest scores as a set of related indices of achievement. 

HwSwe were not able to control and examine was the differential degree of 
Wining the program teachers had, nor the degree of transfer of the program ^prm- 
ciples » thectesroom setting by the individual teachen includedm tte smdy^We 
Sow however, from the teacher and principal interview data and from ^totto*-*? 
SjSs. mat most teachers trained a, each of the five schools uatodwhM 
they had learned in their Consistency Management sessions In add-on, ^because an 
teacher ID numbers were not available, grade by school cell sues did not allow for 
comparing student perfoimance at these levels. 



TEACHER INTERVIEWS 

Introduction 



Bearing in October of 1987, a series of structured interviews was 
all rfme principals and a volunteer sample of thirty-five teachers ftom die five 
sSr^vdunteering teachers, four were selected from each of the schools to 
reflect different grade levels and subject areas. Nineteen teachers participated in a 
fiSiS^ws in the fall of 1987; a second round of interviews was held in 
&e sS of 1988 with the five principals and the sample of teachers (less one who 
iTkC available). Tbe interviews ^^^^J^^t 
were conducted by two doctoral students (who are authors on this paper). As a result. 
fcS-^TmJiews were conducted with principals and teachers of the five 
X>u during the 1987-88 year. The first part of this section of teacher imemews 
£S oSher responsesuhe second part presents principal responses.The inter- 
SwTexTlore many relevant factors but here we restrict ourselves to material on the 
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role of inservice and consistfjicy management in improving teaching effectiveness 
and student learning. 

Inservke Programs 

To encourage the teachers' professional development, a large number of organiza- 
tions, inside and outside of the district, present a wide range of inservice programs 
each year. When asked abcut the nature and value of these programs, teachers could 
identify titles, sponsorship, and formats, but could tell little of their content and less 
of their value. Teachers indicated tiut valuable inservice programs have recogniz- 
able characteristics. These included being taught something new or reinforcing ideas 
they had learned before; an interesting speaker or presenter modeled effective in- 
struction by using a variety of teaching techniques; inservice content was relevant to 
their classroom situation, weie easily implemented and gave teachers observable 
signs of success. The immediate demands of the teaching conditions allowed little 
room for higher level professional development 

Consistency Management 

Teachers responded positively to Consistency Management, both as an inservice 
program and as an approach to classroom management. Teacher participation in the 
inservice sessions was voluntary; nonetheless, the principals' encouragement and the 
fostered coUeagueality through the sharing of food at each session maintained a high 
attendance rate in all five schools throughout the year. Teachers did express their 
reluctance to the interviewers at attending the initial presentation in the spring of 
1987, some asserting that they had no management problems or resenting that the 
session was scheduled for all day Saturday. In the fall, when sessions were sched- 
uled for the late afternoon, some teachers resented the sessions taking place at the 
end of a long school day. 

The negative comments concerning logistics were overshadowed by the the gen- 
erally positive responses to the content of the sessions. Almost all of the teachers 
acknowledged that they had enjoyed the sessions and had learned something that was 
helpful to their work in the classroom. They praised the Consistency Management 
inservice program for its meeting many of their criteria for a good inservice program 
- good presenter, relevant material, and variety in presentation strategies and trans- 
ferable for their classrooms. It also responded to some of their concerns of profes- 
sional and colleaguial isolation. They particularly appreciated the discussion and 
interaction among the participants and the material pertinent to their work with urban 
at-risk students. One teacher commented that this program was better preparation for 
their working with inner-city at-risk students than anything they had had in college. 
Another said it was "more like a college course" because the sessions were on-going, 
with time between sessions to assimilate the material, experiment with implementa- 
tion and receive feedback at subsequent meetings, 'it didn't leave us hangin," said 
another, appreciating the continuing support from colleagues and presenter at each 
session. 

o Teachers reported that Consistency Management was also effective as a class- 
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room management program. All of them reported that they were in fact using in their 
classrooms what they had learned in the inservice sessions. Their most implemented 
aspect of the program dealt with classroom management techniques. Many said that 
they adapted the system to suit their particular needs, regular classroom teachers 
using group rewards while special education teachers preferred individual rewards. 
They asserted that they liked the Consistency Management program because it was 
easy to implement and had objective criteria for teacher decision-making. It re- 
minds me to be fair, that I'm not judge and jury." said one. They appreciated that it 
offered mem and their students clear signs of success, motivating them and empha- 
sizing positive behavior. They also welcomed the fact that Consistency Management 
involved the entire school staff, including non-teaching personnel, to maintain the 
behavioral expectations in and out of the classroom and the school. It called for 
creating an environment that was constantly encouraging. Negative comments fo- 
cused on one aspect of the program. Some teachers had philosophical opposition to 
the use of extrinsic rewards. Nonetheless, even those teachers had adopted the pro- 
gram in their classrooms and reported success. 

In the second set of interviews, conducted in the second half of the school year, 
teachers said that they were continuing to use the program. More of them had made 
alterations in their procedures, eliminating aspects that did not work for them or 
modifying others to be more effective. They reported changing the frequency of the 
rewards, directing them more often to individuals than to the group, and moving 
from extrinsic to instrinsic incentives. 



PRINCIPAL INTERVIEWS 

Consistency Management , 

Order and discipline in the schools was a primary concern for the principals. Hign 
student mobility (34%-75%) and teacher turnover rate (10%-25%) contributed to a 
lack of order. The principal at Adams Elementary indicated that the program dra- 
matically reduced referrals to the office because it enabled teachers to handle their 
own discipline problems. She credited the Consistency Management program for 
this change by noting that without it "I would not have survived." This was a com- 
mon theme for the schools. Hie principal at Monroe Elementary stated m May of 
1988: "I still haven't seen many kids. 1 bet I haven't seen 14 lads this year, last year 
I ^ 62 " Other schools also indicated a reduction in disciplinary referrals. The 
principal at Thomas Jefferson stated, "Last year we needed more help with disci- 
pline. Now we are doing very well and can move beyond that." 

The principals said the teachers viewed the program as positive. A common 
theme expressed by the principals was the need for the teachers to acquire tools for 
dealing with various student behaviors. One principal stated that the program pro- 
vided a clear plan throughout the school for discipline, making the teachers respon- 
sible for their students' behavior and removing her from her role of disciplinarian, l 
am not a policeman!" she asserted. 
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The need for a support pro gr a m when the program ended was another need stated by 
the principals. Following the last session (February 1988) the principals talked about 
another session for the spring. One principal commented, "More of a support system 
was needed in place after the end of the workshops, perhaps to focus in on what's 
happening at the end of the year." 

Parental Involvement 

Each of the principals discussed their concerns and efforts related to parental in- 
volvement The programs to involve parents included the following: 

* Technology week. The parents learned with their children about computers dur- 
ing Saturday sessions. 

* Report card night One school had parents pick up the child's report card and meet 
with the teachers. Sixty percent of the parents attended. 

* Newsletters. Stories by the students and articles from the teachers were included 
in this form of communication brought home by the students. 

* Language classes for non-English speaking parents. Spanish speaking parents in 
two schools had classes to improve their communication. 

* Shell n. The Shell Oil company sponsored Saturday programs focusing on math 
and science for both parents and their children. 

* Incentive programs. These included: a) post cards that teachers sent home to com- 
municate positive academic achievements or behavior of students, and 
b) certificates to parent(s) related to student achievement. 

* Festival Activities. Each of the schools had events where food was served and the 
community was invited. 

* PTO. Each school worked to improve formal parent organizations. 

One lesson learned from parental and community involvement is that one approach 
will not involve all the parents. A variety of programs and strategies are needed to 
increase total parental involvement The end result is an important message for 
students, which lets them know school is important 

Implications 

There are numerous implications from the perspective of the principals in this study. 
The concepts of school-based management was an unspoken theme which emerged 
from the principal interviews. The five principals in defining their good and bad days 
emphasized the need to be out of the office and in the classrooms. Providing re- 
sources to the schools and developing strategies with their teachers to improve 
achievement were their primary objectives. 

Parental involvement in communities where students may be J'ving with grand- 
parents, other relatives or in dysfunctional families, presents serious problems in the 
partnership each of the principals identified as necessary for educational improve- 
ment The strategies used by the principals and teachers represented a layered ap- 
q proach to parental involvement One strategy may increase parental involvement by 
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5 percent but the implementation of several strategies could boost that involvement 
to forty or fifty percent The use of classroom strategies (e.g., positive post cards 
home and telephone calls for positive home contacts at the beginning of the year) 
combined with school wide strategies (e.g„ report card night, am! technology week) 
increased the opportunities far parents to become aware and supportive of their 

children's education. 

Rather than looking for one strategy to solve all the problems, it is evident from 
the interviews that the principals are looking for a multidimensional approach to ef- 
fective schooling. 
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CONCLUSIONS: THE HALF-LIFE OF IMPROVING SCHOOLS 

The resources provided by the district to these five at-risk schools have enabled them 
to turn the corner and create a positive learning environment for the students and 
orderly and supportive working environment for the teachers and administrators. 

In response to the first workshop, over twenty written comments on the district's 
evaluation related to teacher requests for continuance of the program during the 
summer. As one teacher stated in the comments section, "...please don't leave us 
hanging;* Veteran teachers have seen innovations come and go and they realize (as 
evident in their comments) the importance of long term staff development which 
meets their needs. The school district in this study under the leadership of a new 
superintendent, has made a clear commitment to making the schools a bridge to 
personal and academic success for all students. 

It is not just the students who are at-risk of failing, but the schools which provide 
a safety net between what may be a bleak and difficult existence and a hopeful and 
productive future. The half-life factors of improving schools represent a concept that 
describes the decay which eats away at extraordinary efforts to improve the learning 
environment of students. As long as the community and families' needs are not being 
met by the larger society, then extraordinary efforts will be necessary to maintain a 
level of equilibrium to continue improvement efforts. Once a school begins to im- 
prove and exceeds the criteria for its students being at-risk, district supports may 
begin to be removed and redistributed to other newly identified at-risk schools (Fre- 
chtling, 1987; Freiberg, 1989). Within a relatively short period of time, the school 
begins to be at-risk of not sustaining the delicate equilibrium and safety net that 
provides a climate of growth and learning for students and their teachers. Given 
limited resources, the funding for extra supplies or specialized staff development is 
withdrawn and moved to another set of "at-risk" students and schools. The improv- 
ing efforts begin to decay as the teachers and administrators see both intangible and 
tangible supports being removed. Add to the withdrawal of support, the attrition of 
teachers and the high mobility rates of students, the school becomes "at-risk" again, 
in need of new interventions. Preventing decay in school improvement efforts may 
require sustaining support for as long as home and community conditions create 
students who are at-risk of failure in schools and society. 
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The Temple University Center for Research in Human Development and Education is an interdisciplinary center for 
the study of emerging problems ami challenges lacing children, youth, am! families. lis overall goal is to investigate the basic 
forces that affect human development as well as educational processes and outcomes. An important focus of the Center's work 
is the identification and shaping of effective responses to these forces through far-reaching changes in institutional policies ami 
practices. 

The problems and challenges lacing children, youth, and families stem from a variety of cultural, economic, political, 
and health pressures. Their solutions are, by nature, complex. They require long-term programs of study that apply knowledge 
ami expertise from many disciplines and professions. To this end, the Center draws together the many resources of the 
University and a wide range of national, slate, and regional programs. The result is interdisciplinary and interdepartmental 
collaborations that involve investigations of social, economic, educational, and developmental factors ami demonstration of state- 
of-the-art models for training ami for delivery of relevant services. Research and development projects in these anas reflect 
a commitment to enhance the knowledge base for improving the quality of life for children and families, particularly in urban 
environments. 

The work of the Center for Research in Human Development and Education is divided into four program units: 
improving Instruction and Learning in Schools, which provides technical assistance and training for innovative school programs. 
Social Service Delivery Systems, which develops models for effective social service delivery; Studies of Child Development 
ami Early Intervention, which conducts preschool diagnosis and produces innovative program development; and the National 
Center on Education in the Inner Cities (CEIC), funded by the U.S. Department of Education's Office of Educational Research 
and Improvement, which has undertaken a program of research and development as well as dissemination that takes bold steps 
to mobilize and strengthen education and related resources to foster resilience and teaming success of children, youth, ami their 
families in inner cities. 
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